Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need.

OBJECTIVE: To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning. METHODS: We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (ie, trainable vectors) with frozen LLM, where the LLM parameters were not updated (ie, frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. RESULTS AND CONCLUSION: The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ∼3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4%-10% for clinical abbreviation disambiguation, and 5.5%-9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the "one model for all" promise from training to deployment using a unified generative LLM.

A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey.

In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.

The Virtual Child.

SUMMARY: We are building the world's first Virtual Child-a computer model of normal and cancerous human development at the level of each individual cell. The Virtual Child will "develop cancer" that we will subject to unlimited virtual clinical trials that pinpoint, predict, and prioritize potential new treatments, bringing forward the day when no child dies of cancer, giving each one the opportunity to lead a full and healthy life.

A study of generative large language model for medical research and healthcare.

There are enormous enthusiasm and concerns in applying large language models (LLMs) to healthcare. Yet current assumptions are based on general-purpose LLMs such as ChatGPT, which are not developed for medical use. This study develops a generative clinical LLM, GatorTronGPT, using 277 billion words of text including (1) 82 billion words of clinical text from 126 clinical departments and approximately 2 million patients at the University of Florida Health and (2) 195 billion words of diverse general English text. We train GatorTronGPT using a GPT-3 architecture with up to 20 billion parameters and evaluate its utility for biomedical natural language processing (NLP) and healthcare text generation. GatorTronGPT improves biomedical natural language processing. We apply GatorTronGPT to generate 20 billion words of synthetic text. Synthetic NLP models trained using synthetic text generated by GatorTronGPT outperform models trained using real-world clinical text. Physicians' Turing test using 1 (worst) to 9 (best) scale shows that there are no significant differences in linguistic readability (p = 0.22; 6.57 of GatorTronGPT compared with 6.93 of human) and clinical relevance (p = 0.91; 7.0 of GatorTronGPT compared with 6.97 of human) and that physicians cannot differentiate them (p < 0.001). This study provides insights into the opportunities and challenges of LLMs for medical research and healthcare.

Accelerating artificial intelligence: How federated learning can protect privacy, facilitate collaboration, and improve outcomes.

Cross-institution collaborations are constrained by data-sharing challenges. These challenges hamper innovation, particularly in artificial intelligence, where models require diverse data to ensure strong performance. Federated learning (FL) solves data-sharing challenges. In typical collaborations, data is sent to a central repository where models are trained. With FL, models are sent to participating sites, trained locally, and model weights aggregated to create a master model with improved performance. At the 2021 Radiology Society of North America's (RSNA) conference, a panel was conducted titled "Accelerating AI: How Federated Learning Can Protect Privacy, Facilitate Collaboration and Improve Outcomes." Two groups shared insights: researchers from the EXAM study (EMC CXR AI Model) and members of the National Cancer Institute's Early Detection Research Network's (EDRN) pancreatic cancer working group. EXAM brought together 20 institutions to create a model to predict oxygen requirements of patients seen in the emergency department with COVID-19 symptoms. The EDRN collaboration is focused on improving outcomes for pancreatic cancer patients through earlier detection. This paper describes major insights from the panel, including direct quotes. The panelists described the impetus for FL, the long-term potential vision of FL, challenges faced in FL, and the immediate path forward for FL.

Health system-scale language models are all-purpose prediction engines.

Physicians make critical time-constrained decisions every day. Clinical predictive models can help physicians and administrators make decisions by forecasting clinical and operational events. Existing structured data-based clinical predictive models have limited use in everyday practice owing to complexity in data processing, as well as model development and deployment1-3. Here we show that unstructured clinical notes from the electronic health record can enable the training of clinical language models, which can be used as all-purpose clinical predictive engines with low-resistance development and deployment. Our approach leverages recent advances in natural language processing4,5 to train a large language model for medical language (NYUTron) and subsequently fine-tune it across a wide range of clinical and operational predictive tasks. We evaluated our approach within our health system for five such tasks: 30-day all-cause readmission prediction, in-hospital mortality prediction, comorbidity index prediction, length of stay prediction, and insurance denial prediction. We show that NYUTron has an area under the curve (AUC) of 78.7-94.9%, with an improvement of 5.36-14.7% in the AUC compared with traditional models. We additionally demonstrate the benefits of pretraining with clinical text, the potential for increasing generalizability to different sites through fine-tuning and the full deployment of our system in a prospective, single-arm trial. These results show the potential for using clinical language models in medicine to read alongside physicians and provide guidance at the point of care.

Do Gradient Inversion Attacks Make Federated Learning Unsafe?

Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. This capability makes it especially interesting for healthcare applications where patient and data privacy is of utmost concern. However, recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in FL use-cases where the clients' training involves updating the Batch Normalization (BN) statistics and provide a new baseline attack that works for such scenarios. Furthermore, we present new ways to measure and visualize potential data leakage in FL. Our work is a step towards establishing reproducible methods of measuring data leakage in FL and could help determine the optimal tradeoffs between privacy-preserving techniques, such as differential privacy, and model accuracy based on quantifiable metrics.

A large language model for electronic health records.

There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records (EHRs). Natural language processing (NLP) powered by pretrained language models is the key technology for medical AI systems utilizing clinical narratives. However, there are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters (compared with billions of parameters in the general domain). It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs. In this study, we develop from scratch a large clinical language model-GatorTron-using >90 billion words of text (including >82 billion words of de-identified clinical text) and systematically evaluate it on five clinical NLP tasks including clinical concept extraction, medical relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA). We examine how (1) scaling up the number of parameters and (2) scaling up the size of the training data could benefit these NLP tasks. GatorTron models scale up the clinical language model from 110 million to 8.9 billion parameters and improve five clinical NLP tasks (e.g., 9.6% and 9.5% improvement in accuracy for NLI and MQA), which can be applied to medical AI systems to improve healthcare delivery. The GatorTron models are publicly available at: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og .

Rapid artificial intelligence solutions in a pandemic-The COVID-19-20 Lung CT Lesion Segmentation Challenge.

Artificial intelligence (AI) methods for the automatic detection and quantification of COVID-19 lesions in chest computed tomography (CT) might play an important role in the monitoring and management of the disease. We organized an international challenge and competition for the development and comparison of AI algorithms for this task, which we supported with public data and state-of-the-art benchmark methods. Board Certified Radiologists annotated 295 public images from two sources (A and B) for algorithms training (n=199, source A), validation (n=50, source A) and testing (n=23, source A; n=23, source B). There were 1,096 registered teams of which 225 and 98 completed the validation and testing phases, respectively. The challenge showed that AI models could be rapidly designed by diverse teams with the potential to measure disease or facilitate timely and patient-specific interventions. This paper provides an overview and the major outcomes of the COVID-19 Lung CT Lesion Segmentation Challenge - 2020.

Rapid Artificial Intelligence Solutions in a Pandemic - The COVID-19-20 Lung CT Lesion Segmentation Challenge.

Artificial intelligence (AI) methods for the automatic detection and quantification of COVID-19 lesions in chest computed tomography (CT) might play an important role in the monitoring and management of the disease. We organized an international challenge and competition for the development and comparison of AI algorithms for this task, which we supported with public data and state-of-the-art benchmark methods. Board Certified Radiologists annotated 295 public images from two sources (A and B) for algorithms training (n=199, source A), validation (n=50, source A) and testing (n=23, source A; n=23, source B). There were 1,096 registered teams of which 225 and 98 completed the validation and testing phases, respectively. The challenge showed that AI models could be rapidly designed by diverse teams with the potential to measure disease or facilitate timely and patient-specific interventions. This paper provides an overview and the major outcomes of the COVID-19 Lung CT Lesion Segmentation Challenge - 2020.

Generalized chest CT and lab curves throughout the course of COVID-19.

A better understanding of temporal relationships between chest CT and labs may provide a reference for disease severity over the disease course. Generalized curves of lung opacity volume and density over time can be used as standardized references from well before symptoms develop to over a month after recovery, when residual lung opacities remain. 739 patients with COVID-19 underwent CT and RT-PCR in an outbreak setting between January 21st and April 12th, 2020. 29 of 739 patients had serial exams (121 CTs and 279 laboratory measurements) over 50 ± 16 days, with an average of 4.2 sequential CTs each. Sequential volumes of total lung, overall opacity and opacity subtypes (ground glass opacity [GGO] and consolidation) were extracted using deep learning and manual segmentation. Generalized temporal curves of CT and laboratory measurements were correlated. Lung opacities appeared 3.4 ± 2.2 days prior to symptom onset. Opacity peaked 1 day after symptom onset. GGO onset was earlier and resolved later than consolidation. Lactate dehydrogenase, and C-reactive protein peaked earlier than procalcitonin and leukopenia. The temporal relationships of quantitative CT features and clinical labs have distinctive patterns and peaks in relation to symptom onset, which may inform early clinical course in patients with mild COVID-19 pneumonia, or may shed light upon chronic lung effects or mechanisms of medical countermeasures in clinical trials.

Federated learning improves site performance in multicenter deep learning without data sharing.

OBJECTIVE: To demonstrate enabling multi-institutional training without centralizing or sharing the underlying physical data via federated learning (FL). MATERIALS AND METHODS: Deep learning models were trained at each participating institution using local clinical data, and an additional model was trained using FL across all of the institutions. RESULTS: We found that the FL model exhibited superior performance and generalizability to the models trained at single institutions, with an overall performance level that was significantly better than that of any of the institutional models alone when evaluated on held-out test sets from each institution and an outside challenge dataset. DISCUSSION: The power of FL was successfully demonstrated across 3 academic institutions while avoiding the privacy risk associated with the transfer and pooling of patient data. CONCLUSION: Federated learning is an effective methodology that merits further study to enable accelerated development of models across institutions, enabling greater generalizability in clinical use.

Federated Learning used for predicting outcomes in SARS-COV-2 patients.

'Federated Learning' (FL) is a method to train Artificial Intelligence (AI) models with data from multiple sources while maintaining anonymity of the data thus removing many barriers to data sharing. During the SARS-COV-2 pandemic, 20 institutes collaborated on a healthcare FL study to predict future oxygen requirements of infected patients using inputs of vital signs, laboratory data, and chest x-rays, constituting the "EXAM" (EMR CXR AI Model) model. EXAM achieved an average Area Under the Curve (AUC) of over 0.92, an average improvement of 16%, and a 38% increase in generalisability over local models. The FL paradigm was successfully applied to facilitate a rapid data science collaboration without data exchange, resulting in a model that generalised across heterogeneous, unharmonized datasets. This provided the broader healthcare community with a validated model to respond to COVID-19 challenges, as well as set the stage for broader use of FL in healthcare.

CT and clinical assessment in asymptomatic and pre-symptomatic patients with early SARS-CoV-2 in outbreak settings.

OBJECTIVES: The early infection dynamics of patients with SARS-CoV-2 are not well understood. We aimed to investigate and characterize associations between clinical, laboratory, and imaging features of asymptomatic and pre-symptomatic patients with SARS-CoV-2. METHODS: Seventy-four patients with RT-PCR-proven SARS-CoV-2 infection were asymptomatic at presentation. All were retrospectively identified from 825 patients with chest CT scans and positive RT-PCR following exposure or travel risks in outbreak settings in Japan and China. CTs were obtained for every patient within a day of admission and were reviewed for infiltrate subtypes and percent with assistance from a deep learning tool. Correlations of clinical, laboratory, and imaging features were analyzed and comparisons were performed using univariate and multivariate logistic regression. RESULTS: Forty-eight of 74 (65%) initially asymptomatic patients had CT infiltrates that pre-dated symptom onset by 3.8 days. The most common CT infiltrates were ground glass opacities (45/48; 94%) and consolidation (22/48; 46%). Patient body temperature (p < 0.01), CRP (p < 0.01), and KL-6 (p = 0.02) were associated with the presence of CT infiltrates. Infiltrate volume (p = 0.01), percent lung involvement (p = 0.01), and consolidation (p = 0.043) were associated with subsequent development of symptoms. CONCLUSIONS: COVID-19 CT infiltrates pre-dated symptoms in two-thirds of patients. Body temperature elevation and laboratory evaluations may identify asymptomatic patients with SARS-CoV-2 CT infiltrates at presentation, and the characteristics of CT infiltrates could help identify asymptomatic SARS-CoV-2 patients who subsequently develop symptoms. The role of chest CT in COVID-19 may be illuminated by a better understanding of CT infiltrates in patients with early disease or SARS-CoV-2 exposure. KEY POINTS: • Forty-eight of 74 (65%) pre-selected asymptomatic patients with SARS-CoV-2 had abnormal chest CT findings. • CT infiltrates pre-dated symptom onset by 3.8 days (range 1-5). • KL-6, CRP, and elevated body temperature identified patients with CT infiltrates. Higher infiltrate volume, percent lung involvement, and pulmonary consolidation identified patients who developed symptoms.

Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets.

Chest CT is emerging as a valuable diagnostic tool for clinical management of COVID-19 associated lung disease. Artificial intelligence (AI) has the potential to aid in rapid evaluation of CT scans for differentiation of COVID-19 findings from other clinical entities. Here we show that a series of deep learning algorithms, trained in a diverse multinational cohort of 1280 patients to localize parietal pleura/lung parenchyma followed by classification of COVID-19 pneumonia, can achieve up to 90.8% accuracy, with 84% sensitivity and 93% specificity, as evaluated in an independent test set (not included in training and validation) of 1337 patients. Normal controls included chest CTs from oncology, emergency, and pneumonia-related indications. The false positive rate in 140 patients with laboratory confirmed other (non COVID-19) pneumonias was 10%. AI-based algorithms can readily identify CT scans with COVID-19 associated pneumonia, as well as distinguish non-COVID related pneumonias with high specificity in diverse patient populations.

Repair of a duplicate mitral valve in a patient with ascending aortic aneurysm and bicuspid aortic valve.

We present the case of a 48-year-old man presenting to the emergency department with acute onset of chest pain radiating to the back in the setting of a known ascending aortic aneurysm. Preoperative echocardiography revealed an incompetent duplicate mitral valve, with a medially directed eccentric 3+ jet of mitral regurgitation, and a bicuspid aortic valve. The patient was scheduled for aortic root and valve replacement with concurrent mitral valve repair. Intraoperative examination revealed an incompetent duplicate mitral valve separated from the physiologic mitral valve by a band of interatrial tissue. Successful repair was undertaken by oversewing the mitral valve. Postoperative echocardiography allowed for evaluation of this unique repair, which revealed no evidence of further mitral regurgitation and confirmed a successful repair of the mitral regurgitation without any need for an implantable device. Our goal is that this case and its positive treatment outcome, along with our three-dimensional echocardiographic images correlated with the intraoperative findings and photographs, will further the understanding of the diagnosis and treatment of this rare condition.

Janus kinase 3 inhibition with CP-690,550 prevents allograft vasculopathy.

Janus kinase 3 (JAK3) mediates signal transduction from cytokine receptors using the common gamma chain. The rationally designed inhibitor of JAK3, CP-690,550, prevents acute allograft rejection in rodents and in nonhuman primates. Here we investigated the ability of CP-690,550, to prevent allograft vasculopathy in a rodent model of aorta transplantation. Aortas from AxC Irish (RT1(a)) or Lewis (RT1(l)) rats were heterotopically transplanted into the infra-renal aorta of Lewis recipients and harvested at 28 or 56 days. Treated recipients received CP-690,550 by osmotic pumps (mean drug exposure of 110 +/- 38 ng/ml). Significant intimal hyperplasia was demonstrated in untreated allografts when compared with isografts at 28 days (2.08 +/- 0.85% vs. 0.43 +/- 0.2% luminal obliteration, respectively, P = 0.001) and 56 days (5.3 +/- 2.4% vs. 0.38 +/- 0.3%, P = 0.002). Treatment caused a 51% reduction in intimal hyperplasia at day 56. CP-690,550-treated animals also had a significant reduction of donor-specific IgG production and of the gene expression for suppressor of cytokine signaling-3 and with unchanged levels of expression of RANTES, IP-10 and transforming growth factor-beta1. These results are the first to show that JAK3 blockade by CP-690,550 effectively prevents allograft vasculopathy in this rat model of aorta transplantation.

Combined use of the JAK3 inhibitor CP-690,550 with mycophenolate mofetil to prevent kidney allograft rejection in nonhuman primates.

BACKGROUND: Immunosuppression via Janus kinase (JAK) 3 inhibition affords significant prolongation of allograft survival. We investigated the effects of an immunosuppressive regimen combining the JAK3 inhibitor CP-690,550 with mycophenolate mofetil (MMF) in nonhuman primates (NHPs). METHODS: Life-supporting kidney transplantations were performed between ABO-compatible, MLR-mismatched NHPs. Animals were treated orally twice a day with CP-690,550 and MMF (n=8) or MMF alone (n=2) and were euthanized at day 90 or earlier due to allograft rejection. RESULTS: Mean survival time (+/-SEM) in animals treated with MMF alone (23+/-1 days) was significantly extended in animals that concurrently received CP-690,550 (59.5+/-9.8 days, P=0.02). Combination animals exposed to higher levels of CP-690,550 had a significantly better survival (75.2+/-8.7 days) than animals that received less CP-690,550 (33.3+/-12.6 days, P=0.02). Three combination therapy animals were euthanized at day 90 with a subnormal renal function and early-stage acute graft rejection. Rejection, delayed by treatment, ultimately developed in other animals. Anemia and gastrointestinal intolerance was seen in combination therapy animals that otherwise did not show evidence of viral or bacterial infection besides signs consistent with subclinical pyelonephritis (n=3). One incidental lymphosarcoma was noted. CONCLUSIONS: Addition of CP-690,550 to MMF significantly improved allograft survival. The observed side effects appear amenable to improvements upon alteration of dosing strategies. Efficacy of this combination regimen suggests that it could become the backbone of calcineurin inhibitor-free regimens.